Skip to content

[kafka] Serve ApiVersions from registered capabilities - #4260

Draft
gyang94 wants to merge 3 commits into
apache:mainfrom
gyang94:codex/kafka-api-versions
Draft

gyang94 wants to merge 3 commits into
apache:mainfrom
gyang94:codex/kafka-api-versions

Conversation

@gyang94

@gyang94 gyang94 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Purpose

Closes #4265.

Part of umbrella #4185; this PR does not close the umbrella.

Implement ApiVersions negotiation using the capabilities actually registered on the server, and connect the Kafka request entry point to the asynchronous dispatcher. Depends on #4264 and its implementation PR #4259.

Brief change log

  • Add ApiVersionsHandler and build its advertised API keys and version ranges from the capability registry.
  • Replace the request-handler switch and empty stubs with the asynchronous dispatcher.
  • Register only ApiVersions at this stage; Metadata is added in [kafka] Implement Metadata API over Fluss table metadata #4261.
  • Cover supported and unsupported versions, invalid requests, error responses, and advertised capabilities.

Incremental scope: 4 files, +207 / -243 lines, including tests.

Tests

Validated with Java 11 against a temporary checkout whose tracked source files are identical to this rebased head:

  • mvn -o -pl fluss-kafka -am test '-Dtest=Kafka*Test,KafkaRequestITCase' -Dsurefire.failIfNoSpecifiedTests=false: 25 unit tests and 1 integration test passed.
  • Reactor Checkstyle, Spotless, and RAT checks passed; git diff --check passed.
  • 18 additional standalone wire/registry checks passed, covering supported versions, invalid client software fields, unknown tagged fields, unsupported-version fallback and subsequent negotiation, response suppression for acks=0, buffer release, and registry-based capability filtering. These checks are not committed regression tests.
  • git range-diff confirms that the PR02 patch is unchanged after rebasing onto PR01 c557efc22af0b8225f1ebbdd629423cdf409a193.

The entire repository test suite was not run.

API and Format

ApiVersions now reports registered capabilities. No Fluss public API or storage-format change. Produce, Metadata, and topic administration are not implemented in this stage.

Documentation

The supported-capability boundary is documented in this PR and the handler Javadoc.

Stack and review

  1. [kafka] Add request dispatch and transport framework #4259 — request and transport framework.
  2. This PR — ApiVersions.
  3. [kafka] Define DDL table mapping for Kafka compatibility #4277 — DDL table mapping.
  4. [kafka] Implement Metadata API over Fluss table metadata #4261 — Metadata.

This PR remains a draft until its prerequisite lands. The direct base is PR01 c557efc22af0b8225f1ebbdd629423cdf409a193; this PR's head is 5742b36627fe156a8cf499f0aa7b4b8c4163e9a7.

Because the target is Apache main, GitHub's Files changed view includes the prerequisite. For this stage alone, review the incremental comparison. Rebase onto the updated main after the prerequisite is merged and before marking ready.

Generative AI disclosure: Codex assisted with implementation, testing, and commit organization. Human review is required before merge.

gyang94 and others added 3 commits September 16, 2026 16:41
Introduce API registration, request context, version validation, and
asynchronous error mapping. Fix request buffer ownership and response
serialization cleanup while preserving the existing ApiVersions entry point.

Validated with mvn -o -pl fluss-rpc,fluss-kafka verify.

Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 555/555
AI-Contributed/UT: 525/525
Catch failures while mapping asynchronous handler results so the dispatcher
future completes exceptionally instead of remaining pending. Cover handler
futures that fail before and after dispatch registers its callback.

Validated with Java 11: Maven reactor build and 17 targeted Kafka tests,
including Checkstyle, Spotless, and license checks.

Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 27/27
AI-Contributed/UT: 28/28
Route requests through the dispatcher and advertise only implemented APIs.
Return version-aware errors for unsupported APIs and invalid requests.

Validated with mvn -o -pl fluss-kafka verify (23 unit tests and 1 IT).

Co-Authored-By: Codex <noreply@openai.com>
AI-Model: gpt-6
AI-Contributed/Feature: 313/313
AI-Contributed/UT: 137/137
@gyang94
gyang94 force-pushed the codex/kafka-api-versions branch from 0ea02c7 to 5742b36 Compare September 17, 2026 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[kafka] Implement registry-backed ApiVersions API

1 participant